Sampling frame defines units / population is understood
Probabilistic sampling of units
Unbiased estimators guaranteed!
Randomness induced by sampling process
Minimal assumptions!
Typically simple parameters
Model-based
Strong theoretical basis
Population quantities are a random realization of a stochastic generating process; ‘superpopulation’(\(\xi\))
Sampling frame and prob. sampling are not necessary
“no longer looking for a representation of a real population; looking for a mathematical structure or construction that can explain the distribution of a variable and the relationships between variables” - Raymond and Darsaut, 2025.
Model-based
Unbiased estimators are not guaranteed
Randomness is implicit in the process and assumed to follow stochastic distribution
Some-to-very many assumptions
Highly flexible / Basically Dark Magic!
Two types of Inference
Know your population by defining a sampling frame
Think about non random samples (not ignorable)
Use statistical models for flexibility
Statistical Modeling (In a nutshell)
Describes stochastic process that could produce the data
Observed data is just one possible
Model + data allows statistical inferences, i.e., infer features of hypothetical data-generating chance process
Statistical modeling: building of model and its analysis using e.g., maximum likelihood or Bayesian posterior inference
What is a statistical model?
An abstraction / simplification / explanation
Every model has a goal
enforce clarity of thought
summarize
search for patterns
understand mechanisms
predict
What is a statistical model?
“We are rarely explicit about the goals of our models…though should be!”
Is my model good ?
Inference vs prediction
Parameteric Models
Statistical Modeling Concerns
Many hypothetical data-generating mechanisms of stochastic process
leads to “messiness” of model selection
When sampling a large fraction of a finite population
Connect random variables, probabilities, and parameters
define prob. functions
discrete and continuous random variables
use/plot prob. functions
notation!
Probability/Statistics
Probability and statistics are the opposite sides of the same coin.
To understand statistics, we need to understand probability and probability functions.
The two key things to understand this connection is the random variable (RV) and parameters (e.g., \(\theta\), \(\sigma\), \(\epsilon\), \(\mu\)).
Motivation
Why learn about RVs and probability math?
Foundations of:
linear regression
generalized linear models
mixed models
Our Goal:
conceptual framework to think about data, probabilities, and parameters
mathematical connections and notation
Not Random Variables
\[
\begin{align*}
a =& 10 \\
b =& \text{log}(a) \times 12 \\
c =& \frac{a}{b} \\
y =& \beta_0 + \beta_1\times c
\end{align*}
\]
All variables here are scalars. They are what they are and that is it. scalars.
Scalars are quantities that are fully described by a magnitude (or numerical value) alone.
Random Variables
\[
y \sim f(y)
\]
\(y\) is a random variable which may change values each observation; it changes based on a probability function, \(f(y)\).
The tilde (\(\sim\)) denotes “has the probability distribution of”.
Which value (y) is observed is predictable. Need to know parameters (\(\theta\)) of the probability function \(f(y)\).
Random Variables
\[
y \sim f(y|\theta)
\]
where ‘|’ is read as ‘given’.
Toss of a coin Roll of a die Weight of a captured elk Count of plants in a sampled plot
Random Variables
\[
y \sim f(y)
\]
The values observed can be understand based on the frequency within the population or presumed super-population. These frequencies can be described by probabilities.
Frequency / Probabilitities
Frequency / Probabilitities
We often only get to see ONE sample from this distribution.
Random Variables
We are often interested in the characteristics of the whole population of frequencies,
central tendency (mean, mode, median)
variability (var, sd)
proportion of the population that meets some condition P(\(8 \leq y \leq\) 12) =0.68
We infer what these are based on our sample (i.e., statistical inference).
Philosophy
Frequentist Paradigm:
Data (e.g., \(y\)) are random variables that can be described by probability distributions with unknown parameters that (e.g., \(\theta\)) are fixed (scalars).
Bayesian Paradigm:
Data (e.g., \(y\)) are random variables (when observed, then fixed) that can be described by probability functions where the unknown parameters (e.g., \(\theta\)) are also random variables that have probability functions that describe them.
Random Variables
\[
\begin{align*}
y =& \text{ event/outcome} \\
f(y|\boldsymbol{\theta}) =& [y|\boldsymbol{\theta}]= \text{ process governing the value of } y \\
\boldsymbol{\theta} =& \text{ parameters} \\
\end{align*}
\]
\(f()\) or [ ] is conveying a function (math).
It is called a PDF when \(y\) is continuous and a PMF when \(y\) is discrete.
PDF: probability density function
PMF: probability mass function
Functions
We commonly use deterministic functions (indicated by non-italic letter); e.g., log(), exp(). Output is always the same with the same input. \[
\hspace{-12pt}\text{g} \\
x \Longrightarrow\fbox{DO STUFF
} \Longrightarrow \text{g}(x)
\]
\[
\hspace{-14pt}\text{g} \\
x \Longrightarrow\fbox{+7
} \Longrightarrow \text{g}(x)
\]
\[
\text{g}(x) = x + 7
\]
Random Variables
Probability: Interested in \(y\), the data, and the probability function that “generates” the data. \[
\begin{align*}
y \leftarrow& f(y|\boldsymbol{\theta}) \\
\end{align*}
\]
Statistics: Interested in population characteristics of \(y\); i.e., the parameters,
\[
\begin{align*}
y \rightarrow& f(y|\boldsymbol{\theta}) \\
\end{align*}
\]
Probability Functions
Special functions with rules to guarantee our logic of probabilities are maintained.
Discrete RVs
\(y\) can only be a certain set of values.
\(y \in \{0,1\}\)
0 = dead, 1 = alive
\(y \in \{0, 1, 2, ..., 15\}\)
count of pups in a litter; max could by physiological constraint
These sets are called the sample space (\(\Omega\)) or the support of the RV.
PMF
\[
f(y) = P(Y=y)
\]
Data has two outcomes (0 = dead, 1 = alive)
\(y \in \{0,1\}\)
There are two probabilities
\(f(0) = P(Y=0)\)
\(f(1) = P(Y=1)\)
PMF
Axiom 1: The probability of an event is greater than or equal to zero and less than or equal to 1.
\[
0 \leq f(y) \leq 1
\] Example,
\(f(0) = 0.1\)
\(f(1) = 0.9\)
PMF
Axiom 2: The sum of the probabilities of all possible values (sample space) is one.
Read this as “the integral of the PDF between 120 and infinity (on the left-hand side) is equal to the probability that the outcome of the random variable is between 120 and infinity (on the right-hand side)”.
The sum of the probability densities of all possible outcomes is equal to 1.
Normal Distribution (PDF Code)
y =rnorm(1000, mean =20, sd =3)hist(y,freq=FALSE,ylim=c(0,0.14))lines(density(y),lwd=3,col=4)curve(dnorm(x, mean=20, sd =3),xlim=c(0,40),lwd=4,col=3,add=TRUE)
Moments
Properties of all probability functions.
1\({^{st}}\) moment is central tendency
2\({^{nd}}\) moment is the dispersion
…
Normal Distribution: parameters (\(\mu\) and \(\sigma\)) are 1\({^{st}}\) and 2\({^{nd}}\) moments
All the evidence/information in a sample (\(\textbf{y}\), i.e., data) relevant to making inference on model parameters (\(\theta\)) is contained in the likelihood function.
“Conceptually simple, but in practice challenging for ecologists”
The pieces
The sample data, \(\textbf{y}\)
A probability function for \(\textbf{y}\):
\(f(\textbf{y};\theta)\) or \([\textbf{y}|\theta]\) or \(P(\textbf{y}|\theta)\)
the unknown parameter(s) (\(\theta\)) of the probability function
We can do this because we are assuming knowing one observation does not tell us any new information about another observation.
\(P(y_{2}|y_{1}) = P(y_{2})\)
Code
Translate the math to code…
# penguin height data y =c(4.34, 3.53, 3.75)# Joint likelihood of mu=3, sigma =1, given our dataprod(dnorm(y, mean =3,sd =1))
[1] 0.01696987
Optimization Code (Grid Search)
Calculate likelihood of many guesses of \(\mu\) and \(\sigma\) simultaneously,
# The Guesses mu =seq(0,6,0.05) sigma =seq(0.01,2,0.05) try =expand.grid(mu,sigma)colnames(try) =c("mu","sigma")# functionfun =function(a,b){prod(dnorm(y,mean = a, sd = b)) }# mapply the function with the inputs likelihood =mapply(a = try$mu, b = try$sigma, FUN=fun)
MLE Code
# maximum likelihood of parameters try[which.max(likelihood),]
mu sigma
925 3.85 0.36
Lets compare to,
sum(y)/length(y)
[1] 3.873333
Likelihood plot (3D)
Sample Size
What happens to the likelihood if we increase the sample size to N=100?
